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The Vanderbilt Summer Research Program in diabetes, 
which was designed to interest medical students in research careers 
and diabetes care, was evaluated. The program provides stipends to 20 
sophomore and junior medical students for 12 weeks of 
preceptor-supervised laboratory* research work, clinical experience, 
and classroom instruction. The evaluation design includes the 
Discrepancy Evaluation Model, which describes differences between 
program standards and realities. A second evaluation focus compares - 
-the long-term effect on program participants vs. a sample of 
{ nonparticipant applicants. Results indicate th6t the program has 
successfully attracted students naive to biomedical research. All 
students reported adequate research supervision and instruction, 
although there was significant variation in the extept to which 
students, preceptors, and other lab personnel collaborated. Students 
devoted the vast majority of time to conducting research, spending 
less time in the classroom and very little in the. clinic. All 
students showed significant pre-post gains on tests that sampled 
knowledge about diabetes and research design. Finally, in comparison 
to nonparticipant s , participants have published and presented more 
research papers, and are more interested iir pursuing careers in 
diabetes care. (Author/SW) 
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Davis ara^telley (1) surveyed the factors associated with successful 
clinical scientists' decisions to pursue research careers. They recommended 
that research opportunities for medical students be promoted. This paper 
describes an ongoing evaluation of one such training, program. 

The goal of . the Vanderbilt University Summer Research Program in Diabetes, 
operating since 1975, is to help reverse v tfie national decline in the number 
of medical students who choose research careers. A secondary goal is tQ 
interest the' participants in diabetes care. The program, provides stipends 

r 

for twenty rising sophomore and junior medical students from Vanderbilt and 
other medical schools for tweLve weeks of preceptor-supervised laboratory 
research work, clinical experience, and classroom instruction. 

The evaluation design consists of two parts. The first is based on the 
Discrepancy Evaluation Model (2) and^Sescribes differences between program 
standards and realities. The second evaluative focus compares the long-term 
effect on program participants vs. a- sample of non-participant applicants. 

In brief, ,the program has successfully attracted students naive to bio- 
medical research and has provided appropriately challenging projects. All 
students reported adequate research supervision and instruction, although there 
was significant variation in the extent to which students, preceptors, and 
other lab personnel actually collaborated. Students devoted the vast majority 
of time to conducting research, spending less time in the classroom and very 
little in clinic. All students showed significant (]3 < .05) pre-post gains 
on tests which sampled knowledge about diabetes and research design. • 



Finally, program participants have published and presented more research 
papers, intend to spend more time on research T, 10 years from now, ff and more 
intend to pursue careers in diabetes care than members x>f the comparison group. 

The program is too young to draw firm conclusions, but it seems to provide 

** f 

experiences which have rejN^tedly influenced the career decisions of success- 
ful researchers. 
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Does Eaily Research Experience Affect 

Subsequent Career Choice? / 

« 

The purpose of this paper is to present the findings of 2 evaluations 
conducted at Vanderbilt University. The subject of .these evaluations was a 
three month Vanderbilt Summer Research Program. in diabetes. The program gives 
medical studies the opportunity 9 to engage in diabetes research under -the super- 

i 

vision of the prominent faculty members of the Vanderbilt Medical Center 

affiliated with the Diabetes Research and Training Center. The Program was 

established to help reverse the prolonged and systematic decline in the number 

of medical students in this country who choose careers which include clinical 

research. A secondary goal of the program is to interest students in diabetea^ 

i 

care. • % 

The program was begun in 1975. Approximately 20 medical students from 
Vanderbilt and other medical schools' are awarded stipends each year. Most of 
• these participants have oompleted one year of medical school. Less frequently, 
a student will participate who has finished his or her second yea*. 
i- Participants are supervised by Vanderbilt Medical School Faculty members 
who are affiliated with the Diabetes Research and Training Center.* Each student 
does work related teethe 'supervisor's ongoing basic science or clinical research. 
For the most part, students are able to ^lect the prbjedt^and faculty advisors 
with whom they wish to work. Students are not laboratory assistants per se. 
Each participant is responsible for conducting a specific research project, for 
organizing data, and for presenting a paper at the conclusion of the summer at a 
special research seminar. Students* also attend two le&turps each week on a 
variety of topics related to diabetes care and research. 
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The program is based on the premise that medical students are influenced in 

theirxareer choices by two complementary types of experiences. Davi£ and Kelley 

(1982) found that successful clinical researchers* were influenced to, pursue their 

careers by early exposure to their particular field. Secondly , : Paiva et. al. (1982) 
* 

reported that medical students 1 career selections appeared to be strongly 
influenced by their interactions with faculty members. Furthermore, of relevance 

\ • 

to the Vanderbilt Program is^tlSfc finding that interactions with role models were 

* * 
found to most often influence the decisions of students entering less popular 

specialties* 

v The evaluation design* consisted of two parts. The first part, a follow-up 

impact evaluation conducted in 1980, studied the long term, effects df the program 

on past participants. The second focus, a formative evaluation basied on the 

discrepancy evaluation model (Provus, 1971), described the prograni and examined 

the extent to which the program met its own standards. First, the! initial 

follow-up evaluation will be reviewed. It was conducted in 198^by a' medical 

student named Kevin Luidal under the direction of Philip Felts; M.ffi, and James 

* * 
W. Pichert, Ph. D. - . 

The findings should be viewed as preliminary and as a small part of a much 
larger evaluation of this program. The fact is that the outcome measures in which 
we are most interested, i.e., selection of research careers, will not become 
apparent until six to eight years after participation in the program. 

The impact evaluation was conducted as follows. Questionnaires were mailed 
to participants and a 'comparison 'group of nonparticipant applicants from the ^ 
previous five years. Sixty-seven percent ,(n=30) of the former participants, 
72% (n=53) of the comparison group returned completed questionnaires* 



The comparison group consisted of nonparticipant applicants. This design 
is admittedly flawed in that the assignment to the tw^ groups was not random'. 
Nonparticipant applicants were rejected for a variety of reasons. However, all 
students applying to the program are successfully enrolled in highly competitive . 
medical schools and it is assumed that the difference between the quality of 
students iti the two groups is not extreme. Nbnparticipant applicants were 
selected as the comparison group rather than a random group of .medical students 
so as to match the two groups as closely as possible on initial interest in 
diabetes and research. 

The comparisons that I will ireport were all found to be statistically signi- 
ficant at the .05 level using chi-square analyses unless otherwise, stated. 

The results of the follow-up were quite positive. 70% of the experimental 
group, as compared <*to 37% of the comparison group, reported planning to spend a 
significant amount of time doing research during their careers. In addition, 80% 
of the former participants have authored papers or have given research presentations, 
as compared to only 34% of the comparison group. 

It was found that an equal percentage of both groups, 34%, have participated 
in one additional research undertaking. However, 13% of the former participants 
engaged in more than one subsequent project, whereas only 2% of the comparison 
group did -so. This finding is notable, although it did not reach statistical 
significance. 

Finally, former participants were more likely to. conduct diabetes-related 
research. Of those former participants who did one additional research project, 
50% of them did diabetes-related work. Of those former participants who did two 
additional projects^J33% did diabetes-projects. The corresponding figures regarding 
the research activities of the comparison group are 33% and 0%, respectively. 
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The results concerning subsequent research involvement are important to the 
program planners because the interim goal of the program is to heighten students 1 
interest in research so that they will subsequently seek out other research 
.opportunities available to them'. It would probably be unrealistic to expect that 
one, three month research experience in isolation would be able to influence a 
significant number of students 1 career selections*. Rather, the expectation is 

r 1 

that students -will .choose research careers because of the net effefct of several , 
research experiences. The, data presented above indicate that this interim goal 

seems -to have been met. Former participants, do in fact seek out opportunities to 

* * 

be involved in research subsequent to the program, particularly diabetes-related 
research. 

4 

Finally, in termp of the second goal of the program, to increase interest in 

diabetes care, the results were also quite positive* Sixty-one percent of the 

~ y. • ' 

former participants reported planning to specialize in internal medicine, the 

f 

specialty that would include caring for diabetic patients, as compared to 52% of 

the comparison group. Furthermore, and more importantly, of those students going 

into internal medicine, 20% of the former participants plan to subspecialize in 

t 

endocrinology, whereas only 6% of the comparison group reported plans to specialize 
in this area. 

In conclusion, the results of the follow-up evaluation suggest that the 
Vanderb'ilt program raises interest both in diabetes-related research and diabetes 
care. 

Next, the process oriented "DEM" evaluation (Provus, 1971) will be discussed. 
Durxng the summer of 1982, the authors conducted a descriptive evaluation for 
purposes of program improvement, and in order to better describe the attributes of 
the program which led to the positive results of the first follow-up study. 



Data were collected by interviewing program participants at three separate times 
during the summer. Faculty supervisors were interviewed once. Of the 20 pairs 
of students and advisors, only 16 were interviewed: the other 4 pairs did their 
projects in other cities and were not accessible. To* reduce the tendency of 
respondents to giv'e only positive comments about the program, the interviewer 
explained that the goal of the evaluation was to improve the program for future 
participants. It seemed that both faculty members -and students were' uninhibited 
in expressing their opipions* 

Four major components to the program were delineated. The next section 
outlines these components, and describes the extent to which the program met 
its objectives. 

The first major objective of the program is that students should have 

prolonged and positive experiences with their faculty supervisors. It was found 

that there were two types of contact between students and supervisors: fprmal 

and informal. Informal supervision took place when advisors were in the same 

vicinity as the students, but were not directly engaged in advising or overseeing 

the students' particular projects. Seven students received almost continuous 

informal supervision, in addition to two hours per week of formal supervision. . 

^his group will be referred to as 1 group 1. In the second grbup, referred to as 

group 2, were six* students who worked in the same vicinity as other lab personnel, « 

but who received no informal supervision from their official advisers and only 

1-1/2 hours per week of formal consultation. Finally, in group 3 were three * 

students who generally worked alone and therefore did not receive any informal 

supervision. However, these students received a grea^'deal ofr'formal supervision. 

Despite these differences in amount of contact between advisors and -students, no 

student reported receiving inadequate supervision. However, five students said 

^that they were dissatisfied with the lack of personal contact, with their advisors* 

As might be expected, four of these students were from group 2, and one from group 3. 

r\ 
O 



In addition to having close working relationships with their preceptors, it 
is al$o hoped that students have enjoyable experiences working with their 

* 

preceptors. This objective appears to have been met. First of all, all students 

and advisors reported that they were satisfied with their research partners. 

Furthermore, although participants had only completed one year of medical school, 

only two of them reported any difficulty due to lack of sufficient background in 

research. Also, only three advisors would have preferred to work with students 

with mor£ research experience. This is an interesting f inding^ as one might 

\ * 
* expect faculty participants to prefer students with research experience, since 

less time would be required for their training. f 

Finally, 58 of 64 responses to interviewer questions indicated that students 

felt that their projects were creative, well designed, relevant, and interesting. 
The second major objective qf the program is that students st" >uid be provided 

with adequate and informative instruction in diabetes and research methods. 

Students were instructed in two arenas: in the laboratory, and by means of a 

lecture series. 

In terms of laboratory instruction, evaluators did not administer a pre- 
post knowledge test because many types of research projects were done, and thus 
the training of each student was highly individualized. However, students were 
asked "to estimate the amount of instruction time within the laboratories. All 
students reported that advisors and technicians did not simply give them busy 
work, but gave them quite a bit of instruction, and assigned challenging tasks. 
Students reported receiving 3 hours of instruction in research procedures by 
laboratory personnel in an average week. Instruction in diabetes in the laboratories 
%*fis less intensive, averaging 1-1/2 hours per week. However, laboratory instruction 
in diabetes was supplemented by the lecture series. 



It is not clear how much students learned from the formal lecture series. 

In the past, the lecture series has proven to be very worthwhile and informative. 

Students have consistently demonstrated significant pre-post knowledge test gains. 

For the 1982 evaluation, the knowledge test was updated in an attempt to more 

accurately reflect the content material of the series. Secondly, attendance at 

the lectures was no.t mandatory and the average student skipped 5 out of 18 

lectures, generally due to their involvement in the laboratory. The change in 

the knowledge test, and the attendance rate, are assumed to at least partially 

explain why -posttest scores showed little improvement in 1982. The average post 

test^score was 65% as compared to the pretest average of 61%. In effect, only' 

four students showed significant improvements in scores whereas most students' 

» 

scores remained relatively stable. 

Responses to "c&nsumer satisf action 11 questions about the lectures were very 
positive. Students rated a serie^ of lectmres every third week, and 190 of the 
194 responses stated that the lectures were relevant. 17* of the 18 lectures were 
described as being clear and well organized, and 16 as being delivered in an 
interesting and enthusiastic manner. Finally, for the majority of the students, 
14 of the 18 lectures covered basically new information. 

The third major goal of the program is to provide research opportunities to 
medical students who have not had any significant exposure ^to research. 

This objective was clearly met in terms of the research program's recruitment 
procedures. All eleven Vanderbilt participants reported that the program had been 
publicized widely and effectively. Also, all students agreed that the recruitment 
material effectively communicated to the student body that prior research experience 
was not a prerequisite for admission into the program. 
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However, Vanderbilt was much less successful in recruiting out-of-town 
students with little research experience. Three of the four out-of-town partici- 
pants reported that the program description had been interpreted to mean that 
prior research experience would be a principal criterion for acceptance into the 
program. ^ 

The last major objective of the program is that the students have productive 
and successful summers. This standard also held up well in the evaluation. 
Eleven stjudents reported that their data will be published in professional journals 
The other five felt that their data would be the basis for important futyre 
investigations. 

Some preliminary data concerning the extent to which the 1982 program impacted 

y 

on the behaviors and attitudes of the student participants were also collected. 

i * * 

It was found that at least four students expect to continue working with their 
advisors by helping to write research publications. Also, nine students reported 
increased enthusiasm for doing research at* the completion of the program. In 
contrast, two students reported being less likely to engage in subsequent research. 

"To summarize, the Vaftderbilt Summer Student Research Prograir is for the most 
part functioning as intended. However, attempts will be made to increase faculty- 
student contact and to improve attendance at the lecture series, amount other thing 

Not only are the program components in place, but the program has also been 
shown t % 6 produce the desired interum effects. Former participants are more 
actively and successfully engaged in research #nd are more interested in pursuing 
careers in diabetes research and care then their comparison group. Further 
documentation of long-term program impacts is, however, necessary before any 
definitive conclusions can be drawn. 
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